Skip to content

chore(e2e): run Playwright visual tests in Docker (#DS-5311) - #1889

Merged
lskramarov merged 4 commits into
19.xfrom
chore/DS-5311-v19
Aug 13, 2026
Merged

chore(e2e): run Playwright visual tests in Docker (#DS-5311)#1889
lskramarov merged 4 commits into
19.xfrom
chore/DS-5311-v19

Conversation

@lskramarov

Copy link
Copy Markdown
Contributor

No description provided.

Backport of #1845 and #1886 from main.

The screenshot baselines are compared with threshold: 0 and carry no
{platform} suffix, so they belong to one OS and one browser build. Until
now a developer could neither run nor regenerate them off CI, and CI
itself was pinned only by whatever ubuntu-latest meant that week.

Deliberate divergences from main, none of which have a counterpart there:

- the base image is pinned to the v1.55.0-noble digest, matching this
  branch's Playwright rather than main's 1.62.1;
- @playwright/test drops its caret. It already resolved to 1.55.0, so
  nothing installed changes, but tools/e2e/run.js needs an exact version
  to name an image tag, and a patch bump moves the bundled Chromium and
  invalidates every baseline;
- packages/cdk joins the build-context allowlist. 73 files under
  packages/components import @koobiq/cdk/{a11y,keycodes,testing}; on main
  the CDK lives inside components/core and needs no entry;
- the ignore rules land in .eslintignore, since this branch predates the
  flat config;
- everything about the docs smoke suite is dropped - there is no
  e2e:docs script and no playwright.docs.config.ts here.

No baseline is regenerated: all 182 pass unchanged inside the container.
Both captured the page before it had finished assembling itself, and both
were exposed by running the suite in Docker, where a developer machine
drives more workers against one dev server than a 4-vCPU runner does.

code-block reaches highlight.js through a dynamic import and rewrites each
block's innerHTML as it resolves, stamping data-language on the element it
has finished. A screenshot taken before that lands catches the page part
highlighted, which changes the element's height rather than a few pixels
and so reports as "Expected an image 1556px by 3540px, received 1556px by
3232px" - a layout regression, to read it. It is also marked slow(): with
fifteen blocks to highlight and two captures of a ~3500px-tall element it
is the heaviest test in the suite, and under contention it was overrunning
the 15s default and failing on the budget with no screenshot to show why.

icon fetches every glyph from /assets/SVGIcons; an unresolved host occupies
no space, so the whole page reflows as they arrive and the diff is a
horizontal text shift rather than a wrong-looking icon. The open dropdown
made it worse by adding an overlay animation on top, and is dropped here as
it was on main in #1848 - 02-light.png is regenerated for that.

Neither wait is used on E2eIconStateAndStyle: it registers no icon resolver,
renders its icons as font classes, and would never satisfy the assertion.
@lskramarov lskramarov self-assigned this Aug 13, 2026
Copilot AI lite review requested due to automatic review settings August 13, 2026 07:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR makes Playwright visual regression runs reproducible across developer machines and CI by running the suite inside a pinned Playwright Docker image, and updates CI + docs to standardize on that flow.

Changes:

  • Add a Docker-based Playwright runner (Dockerfile + compose + wrapper) and wire it into CI and snapshot-approval workflows.
  • Pin @playwright/test to an exact version and validate Docker image/browser alignment to keep screenshot baselines stable.
  • Reduce visual-test flakiness by waiting for async rendering (icons loading, code highlighting) before taking screenshots.

Reviewed changes

Copilot reviewed 19 out of 21 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
yarn.lock Updates lockfile entries to reflect exact @playwright/test pin.
package.json Pins @playwright/test and adds e2e:docker scripts.
tools/e2e/run.js Adds the local/CI entrypoint wrapper that runs the suite via docker compose.
tools/e2e/Dockerfile Builds a reproducible Playwright+Node+Yarn image aligned with the repo and CI expectations.
tools/e2e/Dockerfile.dockerignore Introduces a strict allowlist build context to keep image builds small and predictable.
tools/e2e/docker-compose.yml Defines the base container run (platform pin, worker cap, output mounts).
tools/e2e/docker-compose.update.yml Adds an overlay for snapshot updates to mount sources back into the working tree.
tools/e2e/assert-browsers.js Fails image build if Playwright browser revisions don’t match the installed Playwright version.
playwright.config.ts Adds validated worker override logic for Docker/CI and documents screenshot stability constraints.
.github/workflows/e2e.yml Runs E2E in Docker (no runner-side browser install) and uploads the report artifact.
.github/workflows/e2e-approve-snapshots.yml Regenerates snapshots inside the same Docker image and commits PNG updates.
docs/guides/06-testing.md Documents Docker-based visual testing workflow and worker behavior.
packages/e2e/README.md Documents Docker requirement/usage for screenshots and snapshot updates.
AGENTS.md Updates contributor guidance to use Docker for visual E2E runs and snapshot updates.
packages/components/icon/e2e.ts Adjusts the icon e2e page to render a dropdown-item inline with the required styles.
packages/components/icon/e2e.playwright-spec.ts Adds a deterministic wait for SVG icon hydration before screenshots.
packages/components/code-block/e2e.playwright-spec.ts Waits for async highlighting and marks the heavy test as slow() for stability.
.eslintignore Ignores Playwright output directories that are produced by local Docker runs.
.prettierignore Ignores Playwright output directories to avoid formatting generated artifacts.
.gitattributes Enforces LF for Docker build inputs and declares PNGs as binary to avoid EOL/filter damage.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tools/e2e/run.js
@github-actions

github-actions Bot commented Aug 13, 2026

Copy link
Copy Markdown

Visit the preview URL for this PR (updated for commit dbbc81b):

https://koobiq-next--prs-1889-ybv5gaa1.web.app

(expires Sun, 16 Aug 2026 10:35:36 GMT)

🔥 via Firebase Hosting GitHub Action 🌎

Sign: c9e37e518febda70d0317d07e8ceb35ac43c534c

@lskramarov lskramarov changed the title chore(e2e): run Playwright visual tests in Docker (#DS-5311) chore(e2e): run Playwright visual tests in Docker (#DS-5311) Aug 13, 2026
docker compose creates a missing bind-mount source itself, but the daemon
does it - so against a rootful daemon playwright-report/ and test-results/
land in the working tree owned by root, and the developer's next run cannot
write into them. It does not heal itself either: the ownership survives
until someone with sudo removes the directories, and the error it eventually
produces comes from inside the container, pointing nowhere near the cause.

Both CI workflows already carried a `mkdir -p` for exactly this, but
tools/e2e/run.js did not, so a plain local run stayed exposed. Creating them
in the wrapper covers every supported entry point, and the workflow steps go
away rather than duplicating it.
Comment thread packages/components/icon/e2e.ts Outdated
// receives the panel's styles through the overlay. Pulling them in here is what keeps it looking
// like a dropdown item — which is the point of the capture, since the icon inside it is what is
// being checked.
styleUrls: ['../dropdown/dropdown-tokens.scss', '../dropdown/dropdown.scss'],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

не совсем честный тест получается, кажется что стоит его просто отрефакторить

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Согласен, убрал блок целиком — dbbc81b.

Честно этот item отрендерить тут нельзя. kbq-dropdown-item живёт только внутри оверлея, поэтому прошлая версия открывала дропдаун и ловила ту часть панели, которая случайно попадала в bounding box компонента: на старом baseline она обрезана правым краем. Плюс съёмка шла наперегонки с анимацией открытия — именно это и делало тест флаки. Вариант с inline-item менял одну подделку на другую: стили панели импортировались в страницу иконок через styleUrls, чтобы кнопка просто выглядела как item.

Внешний вид item с иконкой уже покрыт e2e дропдауна, в настоящей панели. Теряем только иконку, отрезолвленную в inline SVG, а не в font-class — разница слишком тонкая, чтобы держать ради неё любую из двух подделок.

Заодно ушёл KbqDropdownModule из imports страницы, baseline перегенерирован в контейнере.

It could not be rendered truthfully. A kbq-dropdown-item only exists inside
an overlay, so the previous version opened the dropdown and captured
whatever of the panel happened to overlap the component's bounding box -
clipped at the edge, and timed against the open animation, which is what
made the test flaky in the first place. Rendering the item inline instead
traded that for importing the dropdown's own stylesheets into this page to
fake the appearance, which is no more honest.

The appearance of an item with an icon is already covered by the dropdown
suite, in a real panel. What is lost here is the icon resolved as inline SVG
rather than as a font class, which is a thin enough distinction not to be
worth either fake.
@lskramarov
lskramarov merged commit fe12274 into 19.x Aug 13, 2026
9 checks passed
@lskramarov
lskramarov deleted the chore/DS-5311-v19 branch August 13, 2026 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants